gh-142889: Improve layout of dictionary keys by brijkapadia · Pull Request #150640 · python/cpython

brijkapadia · 2026-05-30T21:36:04Z

This is based on the work of @clintonsteiner in PR #145097. Most of it is very similar although I did make my own changes.

This PR updates the memory layout of _dictkeysobject to improve performance.

Issue: Improved performance and arguably simpler code for dictionaries by changing the keys layout #142889

markshannon · 2026-06-02T09:51:33Z

Have you asked @clintonsteiner if it is OK for you to take over their work?

brijkapadia · 2026-06-02T11:20:45Z

@markshannon No, I didn't. I thought it would be okay to create a new PR since the head repo of #145097 was deleted a month ago. However, I can close this if you think this is unfair.

@clintonsteiner Do you want to continue this?

Co-authored-by: Pieter Eendebak <pieter.eendebak@gmail.com>

brijkapadia · 2026-07-23T10:50:02Z

@markshannon, a reminder about this pr, whenever you get a chance.

markshannon · 2026-08-27T12:34:23Z

I guess we aren't going to hear from @clintonsteiner.

@brijkapadia is this all your own work, or did you use @clintonsteiner's code?
If it is your work, I will be happy to merge it.

Do you have any performance numbers for this?

brijkapadia · 2026-08-27T12:55:16Z

This is mainly my work.

While there were many similarities to begin with, after gh-150490, I had to restructure the code.

Right now I'm away from my computer, but I could try to get performance numbers next week.

brijkapadia · 2026-08-31T13:54:47Z

Here are the numbers:

pyperformance benchmarks

ascii85_large

Mean +- std dev: 9.55 ms +- 0.18 ms -> 11.47 ms +- 0.62 ms: 1.20x slower
Significant (t=-23.05)

ascii85_small

Mean +- std dev: 461 us +- 7 us -> 559 us +- 24 us: 1.21x slower
Significant (t=-30.37)

async_generators

Mean +- std dev: 341 ms +- 8 ms -> 371 ms +- 12 ms: 1.09x slower
Significant (t=-15.49)

async_tree_cpu_io_mixed

Mean +- std dev: 454 ms +- 16 ms -> 491 ms +- 17 ms: 1.08x slower
Significant (t=-12.35)

async_tree_cpu_io_mixed_tg

Mean +- std dev: 471 ms +- 17 ms -> 502 ms +- 20 ms: 1.06x slower
Significant (t=-9.00)

async_tree_eager

Mean +- std dev: 87.5 ms +- 2.1 ms -> 93.5 ms +- 4.2 ms: 1.07x slower
Significant (t=-9.91)

async_tree_eager_cpu_io_mixed

Mean +- std dev: 339 ms +- 12 ms -> 356 ms +- 13 ms: 1.05x slower
Significant (t=-7.22)

async_tree_eager_cpu_io_mixed_tg

Mean +- std dev: 425 ms +- 14 ms -> 453 ms +- 18 ms: 1.07x slower
Significant (t=-9.56)

async_tree_eager_io

Mean +- std dev: 590 ms +- 13 ms -> 637 ms +- 21 ms: 1.08x slower
Significant (t=-14.20)

async_tree_eager_io_tg

Mean +- std dev: 617 ms +- 50 ms -> 663 ms +- 53 ms: 1.07x slower
Significant (t=-4.86)

async_tree_eager_memoization

Mean +- std dev: 183 ms +- 9 ms -> 194 ms +- 11 ms: 1.06x slower
Significant (t=-6.17)

async_tree_eager_memoization_tg

Mean +- std dev: 285 ms +- 18 ms -> 301 ms +- 19 ms: 1.05x slower
Significant (t=-4.64)

async_tree_eager_tg

Mean +- std dev: 217 ms +- 12 ms -> 235 ms +- 16 ms: 1.08x slower
Significant (t=-6.87)

async_tree_io

Mean +- std dev: 588 ms +- 15 ms -> 637 ms +- 25 ms: 1.08x slower
Significant (t=-12.88)

async_tree_io_tg

Mean +- std dev: 617 ms +- 40 ms -> 663 ms +- 44 ms: 1.07x slower
Significant (t=-6.05)

async_tree_memoization

Mean +- std dev: 327 ms +- 21 ms -> 346 ms +- 23 ms: 1.06x slower
Significant (t=-4.81)

async_tree_memoization_tg

Mean +- std dev: 312 ms +- 14 ms -> 336 ms +- 15 ms: 1.08x slower
Significant (t=-8.99)

async_tree_none

Mean +- std dev: 245 ms +- 12 ms -> 263 ms +- 18 ms: 1.07x slower
Significant (t=-6.42)

async_tree_none_tg

Mean +- std dev: 255 ms +- 17 ms -> 271 ms +- 21 ms: 1.07x slower
Significant (t=-4.92)

asyncio_tcp

Mean +- std dev: 288 ms +- 8 ms -> 305 ms +- 13 ms: 1.06x slower
Significant (t=-8.98)

asyncio_tcp_ssl

Mean +- std dev: 934 ms +- 9 ms -> 986 ms +- 15 ms: 1.06x slower
Significant (t=-22.76)

asyncio_websockets

Mean +- std dev: 335 ms +- 9 ms -> 359 ms +- 12 ms: 1.07x slower
Significant (t=-12.45)

base16_large

Mean +- std dev: 7.47 ms +- 0.07 ms -> 8.84 ms +- 0.39 ms: 1.18x slower
Significant (t=-26.50)

base16_small

Mean +- std dev: 420 us +- 10 us -> 499 us +- 25 us: 1.19x slower
Significant (t=-23.18)

base32_large

Mean +- std dev: 2.59 ms +- 2.15 ms -> 2.51 ms +- 0.12 ms: 1.03x faster
Not significant

base32_small

Mean +- std dev: 241 us +- 6 us -> 250 us +- 10 us: 1.04x slower
Significant (t=-5.83)

base64_large

Mean +- std dev: 1.99 ms +- 0.05 ms -> 2.13 ms +- 0.07 ms: 1.07x slower
Significant (t=-12.44)

base64_small

Mean +- std dev: 270 us +- 7 us -> 283 us +- 12 us: 1.05x slower
Significant (t=-7.40)

base85_large

Mean +- std dev: 3.34 ms +- 0.06 ms -> 3.85 ms +- 0.15 ms: 1.15x slower
Significant (t=-24.84)

base85_small

Mean +- std dev: 171 us +- 3 us -> 200 us +- 7 us: 1.17x slower
Significant (t=-27.50)

bench_mp_pool

Mean +- std dev: 89.5 ms +- 59.9 ms -> 59.0 ms +- 30.1 ms: 1.52x faster
Significant (t=3.52)

bench_thread_pool

Mean +- std dev: 1.46 ms +- 0.20 ms -> 1.03 ms +- 0.26 ms: 1.43x faster
Significant (t=10.34)

bpe_tokeniser

Mean +- std dev: 3.27 sec +- 0.08 sec -> 3.67 sec +- 0.12 sec: 1.12x slower
Significant (t=-20.89)

chameleon

Mean +- std dev: 10.7 ms +- 0.4 ms -> 11.6 ms +- 0.5 ms: 1.08x slower
Significant (t=-10.04)

chaos

Mean +- std dev: 43.8 ms +- 1.3 ms -> 46.1 ms +- 2.4 ms: 1.05x slower
Significant (t=-6.70)

comprehensions

Mean +- std dev: 12.3 us +- 0.4 us -> 13.4 us +- 0.5 us: 1.09x slower
Significant (t=-14.39)

connected_components

Mean +- std dev: 441 ms +- 4 ms -> 566 ms +- 86 ms: 1.28x slower
Significant (t=-11.32)

coroutines

Mean +- std dev: 16.7 ms +- 0.5 ms -> 18.5 ms +- 0.7 ms: 1.11x slower
Significant (t=-15.39)

coverage

Mean +- std dev: 91.2 ms +- 7.7 ms -> 90.7 ms +- 2.9 ms: 1.00x faster
Not significant

create_gc_cycles

Mean +- std dev: 1.53 ms +- 0.02 ms -> 1.63 ms +- 0.07 ms: 1.06x slower
Significant (t=-10.15)

crypto_pyaes

Mean +- std dev: 54.3 ms +- 2.1 ms -> 59.1 ms +- 2.7 ms: 1.09x slower
Significant (t=-10.87)

deepcopy

Mean +- std dev: 198 us +- 6 us -> 200 us +- 8 us: 1.01x slower
Not significant

deepcopy_memo

Mean +- std dev: 19.7 us +- 0.7 us -> 21.4 us +- 0.7 us: 1.09x slower
Significant (t=-13.07)

deepcopy_reduce

Mean +- std dev: 2.24 us +- 0.07 us -> 2.28 us +- 0.12 us: 1.02x slower
Not significant

deltablue

Mean +- std dev: 2.46 ms +- 0.08 ms -> 2.60 ms +- 0.13 ms: 1.06x slower
Significant (t=-7.28)

docutils

Mean +- std dev: 1.85 sec +- 0.05 sec -> 1.96 sec +- 0.04 sec: 1.06x slower
Significant (t=-13.12)

dulwich_log

Mean +- std dev: 44.8 ms +- 1.3 ms -> 48.3 ms +- 2.3 ms: 1.08x slower
Significant (t=-10.09)

fannkuch

Mean +- std dev: 289 ms +- 9 ms -> 314 ms +- 11 ms: 1.09x slower
Significant (t=-13.69)

float

Mean +- std dev: 53.5 ms +- 1.4 ms -> 57.5 ms +- 3.2 ms: 1.08x slower
Significant (t=-8.94)

gc_traversal

Mean +- std dev: 3.19 ms +- 0.07 ms -> 3.56 ms +- 0.13 ms: 1.11x slower
Significant (t=-19.42)

generators

Mean +- std dev: 25.8 ms +- 0.5 ms -> 27.4 ms +- 1.5 ms: 1.06x slower
Significant (t=-7.72)

go

Mean +- std dev: 88.9 ms +- 1.9 ms -> 98.3 ms +- 7.1 ms: 1.11x slower
Significant (t=-10.00)

hexiom

Mean +- std dev: 4.62 ms +- 0.09 ms -> 5.20 ms +- 0.25 ms: 1.13x slower
Significant (t=-16.81)

html5lib

Mean +- std dev: 49.7 ms +- 1.2 ms -> 54.8 ms +- 3.2 ms: 1.10x slower
Significant (t=-11.70)

json_dumps

Mean +- std dev: 6.81 ms +- 0.15 ms -> 7.72 ms +- 0.71 ms: 1.13x slower
Significant (t=-9.75)

json_loads

Mean +- std dev: 18.3 us +- 0.5 us -> 20.7 us +- 1.2 us: 1.13x slower
Significant (t=-13.80)

k_core

Mean +- std dev: 2.23 sec +- 0.09 sec -> 3.11 sec +- 0.24 sec: 1.39x slower
Significant (t=-26.18)

logging_format

Mean +- std dev: 5.09 us +- 0.19 us -> 5.64 us +- 0.39 us: 1.11x slower
Significant (t=-9.83)

logging_silent

Mean +- std dev: 73.9 ns +- 3.2 ns -> 82.0 ns +- 4.3 ns: 1.11x slower
Significant (t=-11.86)

logging_simple

Mean +- std dev: 4.49 us +- 0.14 us -> 4.91 us +- 0.23 us: 1.10x slower
Significant (t=-12.16)

mako

Mean +- std dev: 8.73 ms +- 0.25 ms -> 10.04 ms +- 0.50 ms: 1.15x slower
Significant (t=-17.96)

many_optionals

Mean +- std dev: 593 us +- 14 us -> 645 us +- 41 us: 1.09x slower
Significant (t=-9.23)

mdp

Mean +- std dev: 982 ms +- 26 ms -> 1203 ms +- 87 ms: 1.22x slower
Significant (t=-18.87)

meteor_contest

Mean +- std dev: 81.0 ms +- 2.2 ms -> 90.1 ms +- 4.2 ms: 1.11x slower
Significant (t=-14.99)

nbody

Mean +- std dev: 73.4 ms +- 2.6 ms -> 83.4 ms +- 6.2 ms: 1.14x slower
Significant (t=-11.60)

nqueens

Mean +- std dev: 61.0 ms +- 1.7 ms -> 80.5 ms +- 2.2 ms: 1.32x slower
Significant (t=-54.48)

pathlib

Mean +- std dev: 9.21 ms +- 0.25 ms -> 13.20 ms +- 0.96 ms: 1.43x slower
Significant (t=-31.26)

pickle

Mean +- std dev: 8.94 us +- 0.21 us -> 11.61 us +- 0.39 us: 1.30x slower
Significant (t=-47.31)

pickle_dict

Mean +- std dev: 23.1 us +- 0.6 us -> 30.5 us +- 1.2 us: 1.32x slower
Significant (t=-43.83)

pickle_list

Mean +- std dev: 3.94 us +- 0.12 us -> 5.43 us +- 0.31 us: 1.38x slower
Significant (t=-35.01)

pickle_pure_python

Mean +- std dev: 252 us +- 6 us -> 333 us +- 18 us: 1.32x slower
Significant (t=-33.12)

pidigits

Mean +- std dev: 181 ms +- 3 ms -> 225 ms +- 39 ms: 1.24x slower
Significant (t=-8.80)

pprint_pformat

Mean +- std dev: 1.26 sec +- 0.03 sec -> 1.20 sec +- 0.03 sec: 1.05x faster
Significant (t=10.43)

pprint_safe_repr

Mean +- std dev: 620 ms +- 16 ms -> 585 ms +- 25 ms: 1.06x faster
Significant (t=9.36)

pyflate

Mean +- std dev: 313 ms +- 6 ms -> 307 ms +- 7 ms: 1.02x faster
Significant (t=5.52)

python_startup

Mean +- std dev: 9.70 ms +- 0.22 ms -> 9.63 ms +- 0.20 ms: 1.01x faster
Not significant

python_startup_no_site

Mean +- std dev: 5.78 ms +- 0.12 ms -> 5.77 ms +- 0.14 ms: 1.00x faster
Not significant

raytrace

Mean +- std dev: 224 ms +- 4 ms -> 219 ms +- 6 ms: 1.02x faster
Significant (t=5.91)

regex_compile

Mean +- std dev: 126 ms +- 4 ms -> 121 ms +- 3 ms: 1.04x faster
Significant (t=6.94)

regex_dna

Mean +- std dev: 138 ms +- 4 ms -> 133 ms +- 4 ms: 1.03x faster
Significant (t=6.42)

regex_effbot

Mean +- std dev: 2.10 ms +- 0.06 ms -> 2.11 ms +- 0.04 ms: 1.00x slower
Not significant

regex_v8

Mean +- std dev: 15.8 ms +- 0.4 ms -> 15.5 ms +- 0.4 ms: 1.02x faster
Not significant

richards

Mean +- std dev: 36.4 ms +- 0.8 ms -> 37.0 ms +- 1.3 ms: 1.02x slower
Not significant

richards_super

Mean +- std dev: 42.4 ms +- 1.3 ms -> 42.8 ms +- 1.4 ms: 1.01x slower
Not significant

scimark_fft

Mean +- std dev: 220 ms +- 5 ms -> 228 ms +- 14 ms: 1.04x slower
Significant (t=-4.55)

scimark_lu

Mean +- std dev: 94.1 ms +- 3.8 ms -> 93.5 ms +- 3.5 ms: 1.01x faster
Not significant

scimark_monte_carlo

Mean +- std dev: 49.5 ms +- 1.3 ms -> 51.1 ms +- 1.6 ms: 1.03x slower
Significant (t=-5.75)

scimark_sor

Mean +- std dev: 88.9 ms +- 2.1 ms -> 89.0 ms +- 2.9 ms: 1.00x slower
Not significant

scimark_sparse_mat_mult

Mean +- std dev: 3.28 ms +- 0.08 ms -> 3.28 ms +- 0.20 ms: 1.00x faster
Not significant

shortest_path

Mean +- std dev: 462 ms +- 4 ms -> 630 ms +- 146 ms: 1.36x slower
Significant (t=-8.93)

spectral_norm

Mean +- std dev: 70.8 ms +- 1.9 ms -> 70.3 ms +- 1.5 ms: 1.01x faster
Not significant

sphinx

Mean +- std dev: 6.42 ms +- 0.18 ms -> 6.33 ms +- 0.14 ms: 1.01x faster
Not significant

sqlalchemy_declarative

Mean +- std dev: 96.1 ms +- 2.5 ms -> 94.7 ms +- 1.6 ms: 1.01x faster
Not significant

sqlalchemy_imperative

Mean +- std dev: 13.3 ms +- 0.3 ms -> 13.0 ms +- 0.3 ms: 1.02x faster
Not significant

sqlglot_v2_normalize

Mean +- std dev: 89.1 ms +- 2.0 ms -> 88.1 ms +- 2.8 ms: 1.01x faster
Not significant

sqlglot_v2_optimize

Mean +- std dev: 43.7 ms +- 0.9 ms -> 43.0 ms +- 1.3 ms: 1.02x faster
Not significant

sqlglot_v2_parse

Mean +- std dev: 938 us +- 27 us -> 934 us +- 20 us: 1.00x faster
Not significant

sqlglot_v2_transpile

Mean +- std dev: 1.18 ms +- 0.03 ms -> 1.16 ms +- 0.03 ms: 1.02x faster
Not significant

sqlite_synth

Mean +- std dev: 1.83 us +- 0.08 us -> 1.81 us +- 0.06 us: 1.01x faster
Not significant

subparsers

Mean +- std dev: 7.78 ms +- 0.21 ms -> 8.60 ms +- 0.50 ms: 1.11x slower
Significant (t=-11.69)

telco

Mean +- std dev: 5.93 ms +- 0.18 ms -> 5.75 ms +- 0.16 ms: 1.03x faster
Significant (t=5.63)

tomli_loads

Mean +- std dev: 1.57 sec +- 0.05 sec -> 1.58 sec +- 0.04 sec: 1.01x slower
Not significant

tornado_http

Mean +- std dev: 83.2 ms +- 2.4 ms -> 82.2 ms +- 2.8 ms: 1.01x faster
Not significant

typing_runtime_protocols

Mean +- std dev: 107 us +- 4 us -> 102 us +- 4 us: 1.05x faster
Significant (t=7.14)

unpack_sequence

Mean +- std dev: 35.4 ns +- 1.5 ns -> 37.1 ns +- 2.0 ns: 1.05x slower
Significant (t=-5.18)

unpickle

Mean +- std dev: 11.0 us +- 0.3 us -> 10.6 us +- 0.3 us: 1.04x faster
Significant (t=6.77)

unpickle_list

Mean +- std dev: 3.57 us +- 0.10 us -> 3.48 us +- 0.10 us: 1.03x faster
Significant (t=5.02)

unpickle_pure_python

Mean +- std dev: 175 us +- 5 us -> 172 us +- 4 us: 1.02x faster
Not significant

urlsafe_base64_small

Mean +- std dev: 300 us +- 7 us -> 320 us +- 14 us: 1.07x slower
Significant (t=-9.44)

xdsl_constant_fold

Mean +- std dev: 39.4 ms +- 4.3 ms -> 39.4 ms +- 4.3 ms: 1.00x slower
Not significant

xml_etree_generate

Mean +- std dev: 72.2 ms +- 1.7 ms -> 74.2 ms +- 3.0 ms: 1.03x slower
Significant (t=-4.43)

xml_etree_iterparse

Mean +- std dev: 78.5 ms +- 2.4 ms -> 76.7 ms +- 2.1 ms: 1.02x faster
Significant (t=4.44)

xml_etree_parse

Mean +- std dev: 119 ms +- 3 ms -> 118 ms +- 4 ms: 1.01x faster
Not significant

xml_etree_process

Mean +- std dev: 53.0 ms +- 1.6 ms -> 53.1 ms +- 2.0 ms: 1.00x slower
Not significant

It seems that this feature is a regression.

pythongh-142889: Improve layout of dictionary keys

f7f967e

brijkapadia requested review from a team, AA-Turner, emmatyping, erlend-aasland, itamaro, markshannon and methane as code owners May 30, 2026 21:36

bedevere-app Bot added the awaiting review label May 30, 2026

bedevere-app Bot mentioned this pull request May 30, 2026

Improved performance and arguably simpler code for dictionaries by changing the keys layout #142889

Open

blurb-it Bot and others added 2 commits May 30, 2026 21:37

📜🤖 Added by blurb_it.

66659cf

update union

13b8d7e

brijkapadia marked this pull request as draft May 30, 2026 22:01

bedevere-app Bot removed the awaiting review label May 30, 2026

brijkapadia added 2 commits May 31, 2026 07:10

Update test/gdb

4476844

update sizeof calculation

fce9aa9

brijkapadia marked this pull request as ready for review May 31, 2026 12:15

bedevere-app Bot added the awaiting review label May 31, 2026

eendebakpt reviewed Jun 19, 2026

View reviewed changes

Comment thread Objects/dictobject.c Outdated

Comment thread Objects/dictobject.c Outdated

Comment thread Objects/dictobject.c Outdated

brijkapadia and others added 8 commits June 19, 2026 15:37

Apply suggestions from code review

1c0ee1b

Co-authored-by: Pieter Eendebak <pieter.eendebak@gmail.com>

Merge remote-tracking branch 'upstream/main' into dict-keys

ae9ad1d

fix const test

708bbae

Merge remote-tracking branch 'upstream/main' into dict-keys

3c15497

small changes

3be6705

fix error

398e7c8

Merge remote-tracking branch 'upstream' into dict-keys

a265605

small changes

d11a0c7

Merge remote-tracking branch 'upstream' into dict-keys

8c99391

Uh oh!

Uh oh!

Conversation

brijkapadia commented May 30, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

markshannon commented Jun 2, 2026

Uh oh!

brijkapadia commented Jun 2, 2026

Uh oh!

Uh oh!

Uh oh!

Uh oh!

brijkapadia commented Jul 23, 2026

Uh oh!

markshannon commented Aug 27, 2026

Uh oh!

brijkapadia commented Aug 27, 2026

Uh oh!

brijkapadia commented Aug 31, 2026

ascii85_large

ascii85_small

async_generators

async_tree_cpu_io_mixed

async_tree_cpu_io_mixed_tg

async_tree_eager

async_tree_eager_cpu_io_mixed

async_tree_eager_cpu_io_mixed_tg

async_tree_eager_io

async_tree_eager_io_tg

async_tree_eager_memoization

async_tree_eager_memoization_tg

async_tree_eager_tg

async_tree_io

async_tree_io_tg

async_tree_memoization

async_tree_memoization_tg

async_tree_none

async_tree_none_tg

asyncio_tcp

asyncio_tcp_ssl

asyncio_websockets

base16_large

base16_small

base32_large

base32_small

base64_large

base64_small

base85_large

base85_small

bench_mp_pool

bench_thread_pool

bpe_tokeniser

chameleon

chaos

comprehensions

connected_components

coroutines

coverage

create_gc_cycles

crypto_pyaes

deepcopy

deepcopy_memo

deepcopy_reduce

deltablue

docutils

dulwich_log

fannkuch

float

gc_traversal

generators

go

hexiom

html5lib

json_dumps

json_loads

k_core

logging_format

logging_silent

logging_simple

brijkapadia commented May 30, 2026 •

edited

Loading